Performance in Practical Problem Solving

The quan t i t y of resources tha t an agent expends in so lv ing problems in a given domain is determined by the representations and search control strategies tha t i t employs. The value of i nd iv idua l representations or strategies to the agent is determined by the i r cont r ibu t ion to the resource expenditure. We argue here t ha t in order to choose the component representat ions and strategies appropriate for a par t i cu la r problem domain it is necessary to measure the i r cont r ibu t ion to the resource expenditure on the actual problems the agent faces. Th is is as true for a system designer m a k i n g such choices as it is for an autonomous mechanical agent. We present one way to measure th is cont r ibut ion and give an example in which the measure is used to improve problem solving performance. 1 I n t r o d u c t i o n A p r imary goal of A r t i f i c i a l Intel l igence research is to enable automated agents to have general reasoning abi l i t ies over a wide range of domains. Due to the inheren t computat ional complexity of th is task, system designers make performance choices t ha t trade genera l i ty for improved resource use. Such performance choices are necessary for any agent w i t h Limited amounts of space and t ime to be effective w i t h i n a dynamic wor ld . Most r u n n i n g AI systems have embedded w i t h i n them the choices tha t the i r designers fe l t would maximize the i r usefulness. Un fo r tuna te ly , such choices re ly on hidden assumptions, such as what var ie ty and frequency of problems w i l l be encountered, or now correct or close to op t ima l the eventual solut ion should be. Of ten, the designers themselves may not know wha t these assumptions are, since they are fu r ther obscured, for example, by choices imp l i c i t in the implementat ion of the representat ion language, or by insuf f ic ient pr ior in fo rmat ion on the range of problems tha t the system may encounter. Unfor tunate ly , th is makes i t d i f f i cu l t This work was supported in part by the N IH Public Health Service under grant 1 R01 NS 22407-01, by the National Science Foundation under grant DCR-8602958, and by the A i r Force Systems Command, Rome Air Development Center, Griffiss A i r Force Base, New York 13441-5700, and the A i r Force Office of Scientific Research, Boi l ing AFB, DC 20332 under Contract Number F30602-85-C 0008 which supports the Northeast Ar t i f i c ia l Intell igence Consortium (N AIC). The authors would l ike to thank the Xerox Corporation Univers i ty Grants Program for providing equipment used in the preparation of this paper. for other researchers to determine if the tradeoffs are appropriate for the i r problem domain. In th is paper, we argue tha t performance choices should be made explicit in order to establish measures w i t h which di f ferent choices can be compared. Fur ther , we provide an example of how th is m igh t be done by developing a cost metr ic defined over search control strategies. The shortterm benefit of th is approach is tha t system designers w i l l have a formal ism which can be applied in de termin ing preferred performance choices for the problems they are solv ing. The longte rm benefi t is tha t an automated agent w i l l have the ab i l i t y to evaluate its success in sat is fy ing i ts goals in comparison w i t h other candidate control strategies. 2 I n t r a c t a b i l i t y In te l l igen t agents perform computat ions upon an in te rna l l y stored representation of the wor ld . Examples in [Levesque and Brachman, 1985] show how the choice of representat ion language bears on the computat ional complexi ty of so lv ing problems us ing tha t language. As they point out, reasoning systems " w i l l e i ther be l im i ted in what knowledge tney can represent, or un l im i ted in the reasoning effort they m i g h t requi re." For example, if we use a f irst-order predicate representation language, then there exists no a lgor i thm tha t w i l l decide for any theory T and sentence S encoded in th is language whether S is a theorem of T. However, it m igh t be the case tha t a par t icu lar theory T in wh ich we are interested can be encoded w i t h i n a weaker representat ion, such as f in i te automata. Then there exist a lgor i thms of bounded computat ional complexi ty tha t w i l l answer most questions w i t h regard to this theory, such as if a s t r ing is accepted by the automaton. Unfo r tuna te ly , no a lgor i thm exists tha t decides whether some f irst-order theory can also be expressed in a weaker representat ion language. We w i l l say tha t a problem P is int ractable if there is no expression language L in wh ich to state problem instances of P sucn tha t there is a determinist ic T u r i n g machine tha t can compute solut ions to such instances w i t h i n polynomial t ime and space. The fo l lowing, a version of the t rave l l i ng salesman problem, is an example of a problem believed to be int ractable. Imagine tha t an agent f inds i tsel f in a c i ty and is given a set of n bu i ld ings to v is i t , a long w i t h in format ion as to the distance between each pair of buid ings. The question to determine is whether the agent can v is i t each bu i l d ing exact ly once on a pa th no more than k un i ts long. I t is possible tha t the agent can answer th is question qu ick ly for some given set of parameters, but Hartman and Tenenberg 535 it is believed that there is no deterministic algorithm (serial or parallel) that wi l l solve every possible instance of this problem within an amount of time polynomial in the number of buildings in the problem. Note that this is not a function of the representation, but of the problem, since it is believed that there exists no representation for which this problem is tractable. Many problems that agents are called upon to solve are intractable. The fact that an agent wi l l encounter difficult problems does not mean that the agent should make no attempt to solve them. But in order to perform effectively, an agent must reason not about whether it can solve all possible problems optimally but, rather, how it can maximize resource use over the entire sample of problems that it will encounter. In order to do this, we believe that defining notions of problem class and resource cost is essential.